Context awareness is widely used for mobile solutions which take advantage of information provided by hardware and software available context data, such as mobile device sensors and user's personal information. Hardware and Software available context data are frequently combined in the prior art and they are herein called “Macro context”. Google Now, Google on Tap and Apple Siri are some examples of popular context-aware applications.
Other well-developed prior art technology referred in the present invention is the sound pattern recognition. Services such as Shazam and Sound Hound are able to use short sound samples which can be filtered to perform searches in commercial music recordings databases, allowing users to identify a specific music being played in the radio or ambient.
The present invention proposes that a “sound signature”, representing sound feature also known as “sound features”, and spectrograms be extracted from audible or inaudible sound environment, such as background noise using similar sound recognition technology and that sound signature be logically associated with the prior art macro context information in order to determine a more detailed and refined level of context, herein called “micro context”. Further, a specific micro context can be recognized from reference context database 433 containing labeled micro context entries, by detecting data patterns and regularities. Moreover, that sound and context information can be shared among several different users, allowing richer reference databases and inferences among different users. In addition to data pattern recognition based on existing reference data, also unknown sound signatures and respective micro contexts can be discovered by using unsupervised or semi-supervised learning approaches, and be added to those sound reference databases (sound fingerprint DB). User inputs on regular reports may also be used to support that process of learning new sounds and respective micro contexts. As a result, automatically, mobile device context-aware applications will be able to behave differently according to specific user's micro contexts.
In the classic definition, context awareness is a property of mobile devices that is defined complementarily to “location awareness” and basically provides information about where the user is. The micro context recognition process takes advantage of the prior art macro context information, but by logically associating this macro context with the sound signatures and by detecting patterns and regularities from historical contextual reference information, from this and other users, the present invention allows a deeper level of details about the user context, so that a mobile context-aware application is able to react to what user is doing, how he is doing, where he is in more details (e.g. which meeting room, car, room in the house), what is happening around, who is around or with him.
The sound recognition and micro context recognition processes uses pattern recognition techniques, a branch of machine learning that focuses on the recognition of patterns and regularities in data. Pattern recognition is the assignment of a label to a given input value. Pattern recognition systems are in many cases trained from labeled “training” data (supervised learning), but when no labeled data is available other algorithms can be used to discover previously unknown patterns (unsupervised learning). Therefore, it is also possible discovering new patterns from originally unknown sound signatures and micro context patterns, even if historical reference data is not initially available.
Furthermore, it is possible to discover (and later recognize) a micro context even if the sound cannot be initially identified, even an initially unknown sound, represented by a new discovered sound label may be used as input for the micro context recognition process. Discovered sound and context labels are saved, or “learned”, for further pattern recognition, according to unsupervised learning approaches. The effect is “I don't know what sound is this, but I frequently hear it when I am at a certain place/at a certain day or time/doing something/with someone/using something”.
A special application of the present invention is the possibility of an external speakerphone being deliberately emitting an inaudible synthetic sound sample in a specific environment, in order to emulate new micro contexts. This approach can enable commercial applications, such as tracking visitor's profiles versus specific business locations (a store, a gas station, etc.) and detecting customer's preferences.